-
Notifications
You must be signed in to change notification settings - Fork 8k
Add RTT support to Twister and Pytest #96822
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add RTT support to Twister and Pytest #96822
Conversation
34986da
to
ed7f3b5
Compare
246a4d8
to
994f883
Compare
When the west rtt command is executed, two things happen: - A GDB server is run. - A telnet client (via sockets) is opened to connect to that server. Previously, that telnet client was operating in the line mode in both directions. That mode is perfectly fine for just presenting the captured content (RTT data) to the stdout; however, it is not suitable for interacting with the device via the shell subsystem. None of the control commands, such as tab and arrow keys, were correctly passed down to the device. To solve that the telnet client had to be swithced into the character mode. For that two separate implementations were needed, one for the Unix and one for Windows. Signed-off-by: Marko Sagadin <[email protected]>
994f883
to
ce92fb5
Compare
|
command.append("--runner") | ||
command.append(rtt_runner) | ||
|
||
if rtt_runner in ("jlink", "pyocd"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
probe-rs
runner supports west rtt
as well as --dev-id
if rtt_runner in ("jlink", "pyocd"): | |
if rtt_runner in ("jlink", "pyocd", "probe-rs"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for noticing, I have added probe-rs
.
scripts/pylib/pytest-twister-harness/src/twister_harness/device/device_adapter.py
Outdated
Show resolved
Hide resolved
ce92fb5
to
d9dbc20
Compare
d9dbc20
to
ad1d214
Compare
This commit introduces --device-rtt flag in the Pytest Twister harness, allowing RTT as a communication protocol between the host PC and the DUT. To demonstrate this, a new build configuration was added to the pytest/shell sample. By following the newly added instructions in the scripts/pylib/pytest-twister-harness/README.rst, users can first build the pytest/shell sample with the RTT shell and then run the pytest with specific flags to run the tests via the RTT. Two additional notes related to the changes: - The way the pty process was terminated had to be changed. west rtt command spawns two subprocesses, and the previous implementation only terminated a single one. The already provided terminate_process function was used to terminate all child processes that pty process started. - Previously, exec_command sent out a command string, followed by two newline characters. The first one was meant to execute the command on the device, the second one was meant to produce the next prompt, which would signal that execution of the command finished. In some instances, when RTT was used, the device ignored the second newline, and thus the prompt was not emitted, bringing the function to a halt, until the timeout expired. This was noticed with the `kernel version` command, the one that is used in one of the tests in pytest/shell. Weirdly enough, that was the only instance of the "newline ignoring" that could be found. Testing `kernel uptime`, for example, worked fine. The workaround was to send a command string with a single newline, wait for the command print and then send the second newline. It was confirmed that this approach works both for the UART and RTT. Signed-off-by: Marko Sagadin <[email protected]>
Add support for using RTT as one of the possible communication transports (besides UART and serial pty), when executing tests on hardware with Twister. Users can enable this by: - Adding both --device-testing and --device-rtt flags to the Twister command line, or - Setting 'rtt' to True in the hardware map YAML file. Implementation details: - RTT support uses the west rtt command to connect to the device. - If west rtt: should use a different runner, then --rtt-runner flag or rtt_runner field can be used. - Renaming changes in handlers.py were done to remove the ser/serial prefix from the variables and functions related to pty, since they are now shared by both serial pty and RTT. The current approach using the west rtt command has one limitation: it is currently not possible to test multiple devices that use RTT in parallel. west rtt command will always try a specific set of ports, so running this command while another one is executing doesn't work. Closes: zephyrproject-rtos#57120 Signed-off-by: Marko Sagadin <[email protected]>
ad1d214
to
29a116b
Compare
Summary
This PR fixes shell RTT interactions via
west rtt
command and adds RTT support in Twister and pytest-twister-harness.This enabling device testing via RTT in addition to existing UART and serial pty transports.
Details
I have split this work into 3 separate commits, where each one makes changes to the specific part of the code.
I have tried to capture all the details in the commit messages, so check them out, but an overall view of the PR looks like this:
west rtt
so that is suitable for interacting with the shell subsys. Previously, it was only suitable for presenting logs captured via the RTT, meaning that arrow and navigation keys weren't correctly interpreted. Separate Unix and Windows implementations were needed to resolve this.I have been mostly testing this with the nRF7002dk board.
Below are some commands that I used to test this.
Testing instructions - click me
Run single ZBus test
Uses Uart for communication (see this as a control test group, I want to show that RTT support didn't break anything else).
Uses RTT for communication (
-x
flags add additional flags to the builds, we use them here to enable RTT as logging backend).Run several ZBus tests
They use
console
orztest
harnesses.Uses UART:
Uses RTT:
Run test using the Pytest harness
Uses UART:
Uses RTT (I have added a specific build configuration as a part of the PR work, that's why the
-x
flags are not used here, they are part ofsample.pytest.rtt
:Run a single test using hardware map
west twister --generate-hardware-map map.yaml
.nrf7002dk
, one if you havegp_md_sp1
(I think))platform
andserial
fields, they won't be correct.How the
map.yaml
should look like:How to use it, UART will be used:
To adapt it for RTT copy the map.yaml into map_rtt.yaml and change some fields, see below example:
How to use it, RTT will be used:
Use Pytest directly, without Twister
This way is a bit nicer for development, since you can decide if want to rebuild binary or just correct the Python tests and rerun them.
To test the Pytest via UART use:
Running the above commands should demonstrate:
Limitations
During implementation of RTT support I ran into a problem. Currently it is not possible to specify two or more device in the hardware map. That is because Twister will try to execute tests on the DUTs in the parralel, meaning that several
west rtt
commands need to be able to coexist at once. This would require thewest rtt
command to provide a way to the caller to control all of it's ports, thus preventing any socket clashes.EDIT
I have fixed all failing tests, I have missed them initially. I am not super confident that I have fixed them in correct way nor if I should write more of them that test the added feature. I would be happy to receive to some guidance in regards to this.